The Simplex Method is Strongly Polynomial for the Markov Decision Problem with a Fixed Discount Rate
نویسنده
چکیده
In this short paper we prove that the classic simplex method with the mostnegative-reduced-cost pivoting rule (Dantzig 1947) for solving the Markov decision problem (MDP) with a fixed discount rate is a strongly polynomial-time algorithm. The result seems surprising since this very pivoting rule was shown to be exponential for solving a general linear programming (LP) problem, and the simplex (or simple policy iteration) method with the smallest-index pivoting rule was shown to be exponential for solving an MDP problem regardless of discount rates. As a corollary, the policy-iteration method (Howard 1960) is also a strongly polynomial-time algorithm for solving the MDP with a fixed discount rate.
منابع مشابه
The Simplex and Policy-Iteration Methods Are Strongly Polynomial for the Markov Decision Problem with a Fixed Discount Rate
We prove that the classic policy-iteration method (Howard 1960) and the original simplex method with the most-negative-reduced-cost pivoting rule (Dantzig 1947) are strongly polynomial-time algorithms for solving the Markov decision problem (MDP) with a fixed discount rate. Furthermore, the computational complexity of the policy-iteration and simplex methods is superior to that of the only know...
متن کاملThe simplex method is strongly polynomial for deterministic Markov decision processes
We prove that the simplex method with the highest gain/most-negative-reduced cost pivoting rule converges in strongly polynomial time for deterministic Markov decision processes (MDPs) regardless of the discount factor. For a deterministic MDP with n states and m actions, we prove the simplex method runs in O(nm log n) iterations if the discount factor is uniform and O(nm log n) iterations if e...
متن کاملA polynomial algorithm for linear optimization which is strongly polynomial under certain conditions on optimal solutions
This paper proposes a polynomial algorithm for linear programming which is strongly polynomial for linear optimization problems min{cx : Ax = b, x ≥ 0} having optimal solutions where each non-zero component xj belongs to an interval of the form [αj, αj · 2], where αj is some positive value and p(n) is a polynomial of the number of variables. We do not make any additional assumptions about c and...
متن کاملModeling and Analyzing Incremental Quantity Discounts in Transportation Costs for a Joint Economic Lot Sizing Problem
Joint economic lot sizing (JELS) addresses integrated inventory models in a supply chain. Most of the studies in this field either do not consider the role of the transportation cost in their analysis or consider transportation cost as a fixed part of the ordering costs. In this article, a model is developed to analyze an incremental quantity discount in transportation cost. Appropriate equatio...
متن کاملStrong polynomiality of the Gass-Saaty shadow-vertex pivoting rule for controlled random walks
We consider the subclass of linear programs that formulate Markov Decision Processes (mdps). We show that the Simplex algorithm with the GassSaaty shadow-vertex pivoting rule is strongly polynomial for a subclass of mdps, called controlled random walks (CRWs); the running time is O(|S| · |U |), where |S| denotes the number of states and |U | denotes the number of actions per state. This result ...
متن کامل